Physical Computing and Multimodal Input in Human-computer Interfaces*

The human-computer interface is widely recognized as an important part of any software project. Consequently, the principles of human-computer interaction are increasingly being taught in computer science departments. Usually, this type of course will focus on the design, implementation and testing of graphical user interfaces. Yet today’s ubiquitous computing applications require interfaces that are context aware and respond to more than text from a keyboard or the click of a mouse. Students who have experience developing interfaces with a range of multimodal inputs will be better prepared to meet this growing need. This paper describes a new course that is being taught at two different colleges at the undergraduate level. Students in these classes learn the theory behind good interface design while obtaining practical experience implementing applications that respond to speech, sensor input, and computer vision. This paper also describes how students go on to apply this knowledge to educational applications in subsequent independent projects, working with teachers as clients. Finally, the paper presents several of the projects produced by students in these classes and in subsequent collaborations. CCSC:Northeastern Conference